{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DDUNCwVslByF"
      },
      "source": [
        "# Finetuning\n",
        "\n",
        "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google-deepmind/gemma/blob/main/colabs/finetuning.ipynb)\n",
        "\n",
        "This is an example on fine-tuning Gemma. For an example on how to run a pre-trained Gemma model, see the [sampling](https://gemma-llm.readthedocs.io/en/latest/sampling.html) tutorial.\n",
        "\n",
        "To fine-tune Gemma, we use the [kauldron](https://kauldron.readthedocs.io/en/latest/) library which abstract most of the boilerplate (checkpoint management, training loop, evaluation, metric reporting, sharding,...).\n"
      ]
    },
    {
      "metadata": {
        "id": "gCHTFaE1Hp2z"
      },
      "cell_type": "code",
      "source": [
        "!pip install -q gemma"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "executionInfo": {
          "elapsed": 902,
          "status": "ok",
          "timestamp": 1738170924282,
          "user": {
            "displayName": "",
            "userId": ""
          },
          "user_tz": -60
        },
        "id": "TZQiYQy7EJe3"
      },
      "outputs": [],
      "source": [
        "# Common imports\n",
        "import os\n",
        "import optax\n",
        "import treescope\n",
        "\n",
        "# Gemma imports\n",
        "from kauldron import kd\n",
        "from gemma import gm"
      ]
    },
    {
      "metadata": {
        "id": "U302Kunl3oLu"
      },
      "cell_type": "markdown",
      "source": [
        "By default, Jax do not utilize the full GPU memory, but this can be overwritten. See [GPU memory allocation](https://docs.jax.dev/en/latest/gpu_memory_allocation.html):"
      ]
    },
    {
      "metadata": {
        "id": "Cc8FiomZ3oLu"
      },
      "cell_type": "code",
      "source": [
        "os.environ[\"XLA_PYTHON_CLIENT_MEM_FRACTION\"]=\"1.00\""
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2hWTB015lM0Z"
      },
      "source": [
        "## Data pipeline\n",
        "\n",
        "First create the tokenizer, as it's required in the data pipeline."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "executionInfo": {
          "elapsed": 808,
          "status": "ok",
          "timestamp": 1738170927385,
          "user": {
            "displayName": "",
            "userId": ""
          },
          "user_tz": -60
        },
        "id": "bywIkAHklSlX",
        "outputId": "3eb4674d-d586-4241-f840-8d97a7aec79b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[\u003c_Gemma2SpecialTokens.BOS: 2\u003e, 1596, 603, 671, 3287, 13060]"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "tokenizer = gm.text.Gemma3Tokenizer()\n",
        "\n",
        "tokenizer.encode('This is an example sentence', add_bos=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2_J-Op0DlSNv"
      },
      "source": [
        "First we need a data pipeline. Multiple pipelines are supported including:\n",
        "\n",
        "* [HuggingFace](https://kauldron.readthedocs.io/en/latest/api/kd/data/py/HuggingFace.html)\n",
        "* [TFDS](https://kauldron.readthedocs.io/en/latest/api/kd/data/py/Tfds.html)\n",
        "* [Json](https://kauldron.readthedocs.io/en/latest/api/kd/data/py/Json.html)\n",
        "* ...\n",
        "\n",
        "It's quite simple to add your own data, or to create mixtures from multiple sources. See the [pipeline documentation](https://kauldron.readthedocs.io/en/latest/data_py.html).\n",
        "\n",
        "We use `transforms` to customize the data pipeline, this includes:\n",
        "\n",
        "* Tokenizing the inputs (with `gm.data.Tokenize`)\n",
        "* Creating the model inputs (with `gm.data.Tokenize`))\n",
        "* Adding padding (with `gm.data.Pad`) (required to batch inputs with different lengths)\n",
        "\n",
        "Note that in practice, you can combine multiple transforms into a higher level transform. See the `gm.data.ContrastiveTask()` transform in the [DPO example](https://github.com/google-deepmind/gemma/tree/main/examples/dpo.py) for an example.\n",
        "\n",
        "Here, we try [mtnt](https://www.tensorflow.org/datasets/catalog/mtnt), a small translation dataset. The dataset structure is `{'src': ..., 'dst': ...}`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "executionInfo": {
          "elapsed": 2094,
          "status": "ok",
          "timestamp": 1738170966792,
          "user": {
            "displayName": "",
            "userId": ""
          },
          "user_tz": -60
        },
        "id": "qf3-uXF6n2e0",
        "outputId": "f5c18592-01fe-4d80-ce40-5edb5a08dfbd"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Disabling pygrain multi-processing (unsupported in colab).\n",
            "{\n",
            "    'input': i64[8 200],\n",
            "    'loss_mask': bool_[8 200 1],\n",
            "    'target': i64[8 200 1],\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "ds = kd.data.py.Tfds(\n",
        "    name='mtnt/en-fr',\n",
        "    split='train',\n",
        "    shuffle=True,\n",
        "    batch_size=8,\n",
        "    transforms=[\n",
        "        # Create the model inputs/targets/loss_mask.\n",
        "        gm.data.Seq2SeqTask(\n",
        "            # Select which field from the dataset to use.\n",
        "            # https://www.tensorflow.org/datasets/catalog/mtnt\n",
        "            in_prompt='src',\n",
        "            in_response='dst',\n",
        "            # Output batch is {'input': ..., 'target': ..., 'loss_mask': ...}\n",
        "            out_input='input',\n",
        "            out_target='target',\n",
        "            out_target_mask='loss_mask',\n",
        "            tokenizer=tokenizer,\n",
        "            # Padding parameters\n",
        "            max_length=200,\n",
        "            truncate=True,\n",
        "        ),\n",
        "    ],\n",
        ")\n",
        "\n",
        "ex = ds[0]\n",
        "\n",
        "treescope.show(ex)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3ny2J07G2X7i"
      },
      "source": [
        "We can decode an example from the batch to inspect the model input. We see that the `\u003cstart_of_turn\u003e` / `\u003cend_of_turn\u003e` where correctly added to follow Gemma dialog format."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "executionInfo": {
          "elapsed": 54,
          "status": "ok",
          "timestamp": 1738170967774,
          "user": {
            "displayName": "",
            "userId": ""
          },
          "user_tz": -60
        },
        "id": "Ep2uhBLh07cw",
        "outputId": "1060cdbb-a9d0-438c-9fb4-1cbc658f451e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u003cstart_of_turn\u003euser\n",
            "Would love any other tips from anyone, but specially from someone who’s been where I’m at.\u003cend_of_turn\u003e\n",
            "\u003cstart_of_turn\u003emodel\n",
            "J'apprécierais vraiment d'autres astuces, mais particulièrement par quelqu'un qui était était déjà là où je me trouve.\n"
          ]
        }
      ],
      "source": [
        "text = tokenizer.decode(ex['input'][0])\n",
        "\n",
        "print(text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L_ND9CJDlcSy"
      },
      "source": [
        "## Trainer\n",
        "\n",
        "The [kauldron](https://kauldron.readthedocs.io/en/latest/) trainer allow to train Gemma simply by providing a dataset, a model, a loss and an optimizer.\n",
        "\n",
        "Dataset, model and losses are connected together through a `key` strings system. For more information, see the [key documentation](https://kauldron.readthedocs.io/en/latest/intro.html#keys-and-context).\n",
        "\n",
        "Each key starts by a registered prefix. Common prefixes includes:\n",
        "\n",
        "* `batch`: The output of the dataset (after all transformations). Here our batch is `{'input': ..., 'loss_mask': ..., 'target': ...}`\n",
        "* `preds`: The output of the model. For Gemma models, this is `gm.nn.Output(logits=..., cache=...)`\n",
        "* `params`: Model parameters (can be used to add a weight decay loss, or monitor the params norm in metrics)\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Tvbun7ho7EyV"
      },
      "outputs": [],
      "source": [
        "model = gm.nn.Gemma3_4B(\n",
        "    tokens=\"batch.input\",\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jMb2KpHR7vo5"
      },
      "outputs": [],
      "source": [
        "loss = kd.losses.SoftmaxCrossEntropyWithIntLabels(\n",
        "    logits=\"preds.logits\",\n",
        "    labels=\"batch.target\",\n",
        "    mask=\"batch.loss_mask\",\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S3fXHa_4LnEH"
      },
      "source": [
        "We then create the trainer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Bv854FDSn7Z-"
      },
      "outputs": [],
      "source": [
        "trainer = kd.train.Trainer(\n",
        "    seed=42,  # The seed of enlightenment\n",
        "    workdir='/tmp/ckpts',  # TODO(epot): Make the workdir optional by default\n",
        "    # Dataset\n",
        "    train_ds=ds,\n",
        "    # Model\n",
        "    model=model,\n",
        "    init_transform=gm.ckpts.LoadCheckpoint(  # Load the weights from the pretrained checkpoint\n",
        "        path=gm.ckpts.CheckpointPath.GEMMA3_4B_IT,\n",
        "    ),\n",
        "    # Training parameters\n",
        "    num_train_steps=300,\n",
        "    train_losses={\"loss\": loss},\n",
        "    optimizer=optax.adafactor(learning_rate=1e-3),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xd1RcRekMkRG"
      },
      "source": [
        "Trainning can be launched with the `.train()` method.\n",
        "\n",
        "Note that the trainer like the model are immutables, so it does not store the state nor params. Instead the state containing the trained parameters is returned."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "height": 153,
          "referenced_widgets": [
            "4e5e30e60608416689ede66723bf5d74",
            "6ec6c6c1844a4258ad539f0f4c189491",
            "10f009a6e15d40a1bb2bd81f19a6d8a1",
            "755db625c0c04127a96c54cce3951913",
            "e67e3e0cfafd4b609971fca58e4e530b",
            "d8d6b06e1c834c06a39384d300e9ba8f",
            "df2e64ed38ab4796a86a3fa594d4352b",
            "9cc2c80d152248c4bade0955c25554ad",
            "d8f735a8844d4b3f87bc9c93f0c281e5",
            "e0685ee78f8c499390ac0e1024ab26a9",
            "a022879a38e849cfac7178cea469f36f"
          ]
        },
        "executionInfo": {
          "elapsed": 302084,
          "status": "ok",
          "timestamp": 1738156418198,
          "user": {
            "displayName": "",
            "userId": ""
          },
          "user_tz": -60
        },
        "id": "xvIDsFPz75GT",
        "outputId": "651bc4fb-6a51-431b-fae8-aa0213d1e27b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Configuring ...\n",
            "Initializing ...\n",
            "Starting training loop at step 0\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4e5e30e60608416689ede66723bf5d74",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "train:   0%|          | 0/301 [00:00\u003c?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "state, aux = trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4_VfxhhKRvDd"
      },
      "source": [
        "## Checkpointing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q_y_OgyyM_dT"
      },
      "source": [
        "To save the model params, you can either:\n",
        "\n",
        "* Activate checkpointing in the trainer by adding:\n",
        "\n",
        "  ```python\n",
        "  trainer = kd.train.Trainer(\n",
        "      workdir='/tmp/my_experiment/',\n",
        "      checkpointer=kd.ckpts.Checkpointer(\n",
        "          save_interval_steps=500,\n",
        "      ),\n",
        "      ...\n",
        "  )\n",
        "  ```\n",
        "\n",
        "  This will also save the optimizer, step, dataset state,...\n",
        "\n",
        "\n",
        "* Manually save the trained params:\n",
        "\n",
        "  ```python\n",
        "  gm.ckpts.save_params(state.params, '/tmp/my_ckpt/')\n",
        "  ```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QAbX_hwileZF"
      },
      "source": [
        "## Evaluation\n",
        "\n",
        "Here, we only perform a qualitative evaluation by sampling a prompt.\n",
        "\n",
        "For more info on evals:\n",
        "\n",
        "* See the [sampling](https://gemma-llm.readthedocs.io/en/latest/sampling.html) tutorial for more info on running inference.\n",
        "* To add evals during training, see the Kauldron [evaluator](https://kauldron.readthedocs.io/en/latest/eval.html) documentation.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fOrkpxlAlf2V"
      },
      "outputs": [],
      "source": [
        "sampler = gm.text.ChatSampler(\n",
        "    model=model,\n",
        "    params=state.params,\n",
        "    tokenizer=tokenizer,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x54YaAteRV94"
      },
      "source": [
        "We test a sentence, using the same formatting used during fine-tuning:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "height": 35
        },
        "executionInfo": {
          "elapsed": 8244,
          "status": "ok",
          "timestamp": 1738157670013,
          "user": {
            "displayName": "",
            "userId": ""
          },
          "user_tz": -60
        },
        "id": "yM0l9EnPMdHf",
        "outputId": "d930809e-ae40-4cad-a82f-aed0a3d53aba"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Salut ! Mes vacances suivantes seront à Paris.'"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sampler.chat('Hello! My next holidays are in Paris.')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sPQUGkR3ZcO_"
      },
      "source": [
        "The model correctly translated our prompt to French!\n",
        "\n",
        "## Next steps\n",
        "\n",
        "To fine-tune outside of Colab, you can look at our [examples/](https://github.com/google-deepmind/gemma/tree/main/examples/) folder for more complexes trainer configs, including LoRA, DPO and sharding."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "last_runtime": {},
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "10f009a6e15d40a1bb2bd81f19a6d8a1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9cc2c80d152248c4bade0955c25554ad",
            "max": 301,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d8f735a8844d4b3f87bc9c93f0c281e5",
            "value": 301
          }
        },
        "4e5e30e60608416689ede66723bf5d74": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6ec6c6c1844a4258ad539f0f4c189491",
              "IPY_MODEL_10f009a6e15d40a1bb2bd81f19a6d8a1",
              "IPY_MODEL_755db625c0c04127a96c54cce3951913"
            ],
            "layout": "IPY_MODEL_e67e3e0cfafd4b609971fca58e4e530b"
          }
        },
        "6ec6c6c1844a4258ad539f0f4c189491": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d8d6b06e1c834c06a39384d300e9ba8f",
            "placeholder": "​",
            "style": "IPY_MODEL_df2e64ed38ab4796a86a3fa594d4352b",
            "value": "train: 100%"
          }
        },
        "755db625c0c04127a96c54cce3951913": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e0685ee78f8c499390ac0e1024ab26a9",
            "placeholder": "​",
            "style": "IPY_MODEL_a022879a38e849cfac7178cea469f36f",
            "value": " 301/301 [04:24\u0026lt;00:00,  6.86s/it]"
          }
        },
        "9cc2c80d152248c4bade0955c25554ad": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a022879a38e849cfac7178cea469f36f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d8d6b06e1c834c06a39384d300e9ba8f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d8f735a8844d4b3f87bc9c93f0c281e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "df2e64ed38ab4796a86a3fa594d4352b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e0685ee78f8c499390ac0e1024ab26a9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e67e3e0cfafd4b609971fca58e4e530b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
